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Abstract. In this paper we give topological lower bounds on the numbers of 
periodic and closed trajectories in strictly convex smooth billiards T C R m+1 . 
Namely, for given n we estimate the number of n-periodic billiard trajectories in 
T, and also the number of billiard trajectories, which start and end at a given 
point A G dT and make a prescribed number n of reflections at the boundary dT 
of the billiard domain. We use variational reduction, admitting a finite group 
symmetries, and apply topological approach based on equi variant Morse and 
Lusternik - Schnirelman theories. 



1. Introduction 

Let X C R m+1 be a closed smooth strictly convex hyper surf ace. We will consider 
the billiard system in the (m + l)-dimensional convex body T, bounded by X. 
Recall, that we view the billiard ball as a point, which moves in T in a straight 
line, except when it hits X = dT, where it rebounds, making the angle of incidence 
equal the angle of reflection. 

G.D. Birkhoff [[| studied periodic billiard trajectories in plane convex billiards. 
Papers |7| deal with the problem of estimating the number of periodic trajec- 
tories in convex billiards in R m+1 , where m > 1. In |J we studied the number 
of billiard trajectories having fixed distinct end points and making a prescribed 
number of reflections. 

The purpose of this paper, which continues ||, is twofold. Firstly, we obtain 
estimates on the number of closed billiard trajectories, which start and end at a 
given point A & X and make a prescribed number n of reflections at the hypersur- 
face X. This problem may look as a special case of the fixed end billiard problem 
H, but as we show, presence of symmetry allows to get much stronger estimates 
than ||. Secondly, we give a linear in n estimate on the number of n-periodic 
trajectories. 

The following Theorem [l] gives an estimate on the number of closed billiard 
trajectories. It deals with Z2-orbits of billiard trajectories. Any such Z2-orbit 
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is determined by a sequence of reflection points X\, x 2 , ■ ■ ■ , x n £ X, such that 
Xi 7^ Xi + i for i = l,...,n— 1 and x% ^ A, x n ^ A. The reverse sequence 
x n , x n -i, . . . , x\ determines the same Z 2 -orbit. 

Theorem 1. Let X C R m+1 be a closed smooth strictly convex hypersurface, A £ 
X. 

(I) For any n > 1 £/ie number of distinct ^-orbits of closed billiard trajectories 
inside X , which start and end at A, and make n reflections is at least 

n, ifm>3is odd, 

(1.1) 

[n/2] + 1, if m > 2 is even. 

(II) For any even n > 2 £/ie number of distinct 7^2-orbits of closed billiard tra- 
jectories inside X , which start and end at A, and make n reflections is at least 

[log 2 n] + m — 1 , if m > 3 is odd, 

[log 2 n] + m — 2, if m > 2 is even and n > 4, (1.2) 

m, if m >2 is even and n = 2. 

(III) Ifn>2is even, and the billiard data (X,A,n) is generic (cf. below), then 
the number of distinct Z 2 -orbits of closed billiard trajectories inside X , which start 
and end at A, and make n reflections is at least 

ran/2. (1.3) 

First we explain the genericity assumption in statement (III). The billiard data 
(X, A, n) determines a continuous function 

■n 

X xn ^R, {x l ,...,x n )^^\x i -x i+l \, (1.4) 

i=0 

(the total length), where we understand xq = A = x n+ \. This function is smooth 
at all configurations (x\, . . . , x n ) £ X xn with X{ ^ Xi+\ for i — 0, . . . , n. The data 
(X,A,n) is generic if any critical configuration (xi, . . . ,x n ) £ X xn of the total 
length function (|1.4j) , satisfying the above condition Xi ^ is Morse. Compare 

0,0- 

Statements (I) and (II) give different lower bounds on the number of closed 
billiard trajectories. (I) is linear in n; it is better than (II) for large n. On 
the other hand, (II) may be better than (I) if the dimension ra = dimX of the 
boundary of billiard domain is large. 

Let us compare Theorem [I] with the lower bound on the number of billiard 
trajectories with fixed distinct end points, obtained in 0. In Theorem [I] we speak 
about Z 2 -orbits of billiard trajectories. Each Z 2 -orbit contains one or two billiard 
trajectories. For n even, each Z 2 -orbit contains precisely two distinct billiard 
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trajectories. Hence, we see that for n even statement (I) of Theorem [I] predicts 
twice number of closed billiard trajectories, compared to the estimate of || for the 
billiard trajectories with fixed ends. Also, for large m statements (II) and (III) 
give much larger lower bounds than corresponding estimates of Theorem 1 of ||. 

Statement (III) includes the case m — 1 (the plane billiards) and gives the 
estimate n/2. The billiard in the unit circle has precisely n/2 orbits of closed 
billiard trajectories with a given initial point. 

It is reasonable to expect that for any even n > 2 the number of distinct Z2- 
orbits of closed billiard trajectories inside X, which start and end at A, and make 
n reflections is at least 

n + m — 1, if m > 3 is odd, 

(1.5) 

n/2 + m — 1, if m > 2 is even, 

Such estimate would imply both statements (I) and (II) of Theorem [3]. The meth- 
ods of this paper do not to prove this assertion, although the gap looks very small. 

The proof of Theorem |l] is based on a computation of the cohomology ring of a 
relevant configuration space of points on the sphere S m . We apply the technique 
of the critical point theory, based on the cup-length estimates together with a 
refinement, suggested by E. Fadell and S. Huseini ||, related to the notion of 
category weight of cohomology classes. 

Next we state the main result concerning n-periodic trajectories. 

Theorem 2. Let X C R m+1 be a smooth strictly convex hyper surf ace. For any 
odd prime n there exist at least 

n, if m is odd, , . 

(n+l)/2, if m is even ^ ' ' 

distinct D n -orbits of n-periodic billiard trajectories inside X. 

This theorem complements the results of [J7|. In [^] it is shown that for m > 3 
and odd n the number of distinct -D n -orbits of n-periodic billiard trajectories inside 
X C R m+1 is not less than [log 2 (n — 1)] + m and it at least (n — l)m for generic 
billiards X C R m+1 . These results from are similar to statements (II) and (III) 
of Theorem 0. Theorem |2] above has several advantages compared to . It gives 
a linear in n estimate, which is better for large n than the logarithmic estimate 
of 0. Also, it allows the case m = 2 which corresponds to convex billiards in 
3-dimensional Euclidean space. On the other hand, the result of is better for 
large m. 

The proof of Theorem |2| is based on a computation of the cohomology rings 
of cyclic configuration spaces of spheres with rational coefficients. The case of 
Z2-coefficients was computed in 0. 

I would like to thank S. Tabachnikov for useful discussions. 
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2. Cohomology of the closed string configuration spaces of spheres 

Let 

G n = G(S m ;A,A,n) (2.1) 

denote the closed string configuration space of S m , i.e. the space of all configura- 
tions (xi, . . . , x n ), where Xi G S m , such that X\ ^ A, x n ^ A and x^ ^ x i+ \ for all 
i — 1, . . . ,n — 1. There is a natural involution 

T ■ G n > G n , T(x\ } . . . , x n ) (x n .x n _i, . . . , Xi), (2-2) 

which will be important for the sequel. 

Theorem 3. The cohomology group H % (G n ] Z) is nonzero only in dimensions 

i = o, (m - 1), 2(m - 1), . . . , (n - l)(m - 1) 

and for these values i the group H l (G n ; Z) is free abelian of rank 1. One may 
choose additive generators 

o~i G W^iG^Z), z = 0,l,...,n-l, 

suc/i t/iai /or m > 3 odd, £/ie multiplication is given by 

{U _|_ j\\ 
— — — • <7 i+i , i/ i + j < n - 1, 
* ! ^ ! (2.3) 
0, if i + j>n-l 
and for m > 2 even, z't z's gzi> en fry 

{[(^ + j)/2l! 
r . 1t — F — — - • a i+ A, if i + j < n — 1 and z or ?' is even, 
[</2]! ■ [7/2]! J " (2 _ 4) 

0, i/ either i + j > n — 1, or both i and j are odd. 
Reflection ( \2.3j ) acts for m > 1 odd fry 

T>,) = (-l)Vi, (2.5) 

and /or m > 1 even fry 

T*((Ti) = (-l)^ +ni a t , i = 0, 1, . . . ,n - 1. (2.6) 

Proof. Consider the map 

G„ = G(S' m ; A, A, n) > S"™ — A, (xi, . . . , x n ) i— > x n . 

It is a smooth fibration with fiber G(S m ; A, B,n — 1), where A ^ B. Since the 
base S m — C is contractible, we obtain that the inclusion 

G(S rn ; A,B,n- 1) C G n (2.7) 

is a homotopy equivalence. Hence, the integral cohomology ring of G n coincides 
with H*(G(S m ; A,B,n— 1);Z), which we calculate below. 
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Theorem 8 from || describes algebra H*(G(S m ; A, B,n — l);k), where k is 
an arbitrary field. From this description it is clear that the dimension of the 
cohomology does not depend on field k. Therefore, we conclude that the integral 
cohomology H l (G(S m ; A,B,n— 1); Z) has no torsion; it is a free abelian group of 
rank one for i — r(m — 1), where r = 0, 1, . . . , n — 1, and it vanishes for all other 
values of i. 

Let C G S m be a point distinct from A and B. We obtain an inclusion of 
configuration spaces 0* : G(S m — C; A, B, n — 1) — > G(S m ; A, B, n — 1), where we 
identify S m - C with R m . The cohomology algebra H*(G(K m ; A,B,n — l); Z) has 
generators so, . . . , s n _i and the full list of relations was described in Proposition 7 
of |J . From remark 9 in || we know that the induced map 0* on cohomology with 
arbitrary field of coefficients k is injective. This implies that the induced map on 
integral cohomology 

</>* : H*(G{S m ; A,B,n— 1);Z) — > H*(G(R m ; A,B,n- 1); Z) (2.8) 

is injective and <fi* maps indivisible classes from H*(G(S m ; A, B,n — 1);Z) into 
indivisible classes in H*(G(R m ; A, B, n - 1); Z). 

We claim that for any r = 0,l,...,n — 1 there exists an indivisible class 

a r e H r( ' m - 1 \G(S m ; A,B,n — 1); Z), 

such that 

s h . . . s ir for m odd, 

0<ii<---<i r <ra 

(2.9) 

(_l)[r/2]+nr . £ (_!)«+-+*,- ^ _ _ _ g ^ for m even 

0<ii<— <i r <n 

(compare with formulae (4.3) and (4.4) from ||). Indeed, applying Remark 9 
from H with k = Q, we see that the image of the generator of the group 
H r ( m ~^(G(S m ; A, B,n — 1);Z) ~ Z under homomorphism <p* equals an integral 
multiple of the expression in the RHS of (|2.9|). Since the classes in RHS of ( |2.9| ) 
are indivisible, and since we know that <fi* maps indivisible classes to indivisible 
classes, we conclude that a there exists generator a r with the required property. 

The product formulae ( |2.3j ) and fl2.4| ) for classes a r follow since they hold for the 
products (fi*((Ti)4>((Tj) G H*(G(R m ; A,B,n— 1); Z) as can be easily checked using 
the arguments of the proof of Theorem 8 from || . 

Now we want to find the action of the reflection T : G n — > G n on classes <Tj. It is 
clear that T*(<jj) = ± cr,, and we need to calculate the sign. Consider the following 
diagram of natural inclusions 

G(R m ;A,B,n - 1) -> G(S m ;A,B,n - 1) 

i i 
G(K m ]A,A,n) -> C7(5 m ;A^,"-) 
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(where R m = S m — C as above) and the induced diagram of cohomology groups 

H*(G(R m ; A,A,n);Z) ^ H*(G(S m ; A, A, n); Z) 
PI al~ 
H*(G(R m ; A,B,n-l); Z) t- H*(G(S m ; A,B,n — l); Z) 

a is an isomorphism and <fi* is injective. To understand (3, note that G(R m ; A, A, n) 
is homotopy equivalent to the cyclic configuration space G(R m ,n + 1) (cf. 0) 
and so the cohomology fP(G(R m ; A, A, n); Z) has (m — l)-dimensional generators 
So, Si, . . . , s n which satisfy relations of Proposition 2.2 from [7] (we shift indices 
for convenience). Proof of Proposition 7 in 0] shows that /3(sj) = s« for i = 
0, 1, . . . , n — 1 and /3(s n ) = 0. Hence, (3 is an epimorphism with kernel equal the 
ideal generated by s n . 

The reflection T acts also on G(R m ; A, A, n) (by formula ( ^.2| ). It is clear that 
the induced map T* : H*(G(R m ; A, A, n); Z) -> #*(G(R m ; A, A, n); Z) acts on the 
generators Sj as follows 

T*(si) = (-l) m s n _i, where z = 0,l,...,n. (2.10) 

Now we may calculate T*(cr r ), where r = l,...,n — 1. Fix a subsequence 
< i\ < ■ ■ ■ < i r < n (we avoid indices and n). Suppose first that m is odd. Then 
4>*(a(a r )) contains monomial s^s^ . . . Sj r ; therefore 7(cr r ) contains the same mono- 
mial with coefficient 1. Then T*{^(a r )) contains monomial s n _j r s n _j r _ 1 . . . s n ^i 1 
with coefficient (— l) mr = (— l) r . The last monomial appear in 7(cr r ) with coeffi- 
cient 1. Since we know that T*(a r ) = ±cr r we conclude that T*(a r ) = (— l) r a r . 

Assume now that m is even. Then <p*(a(a r )) contains monomial s^s^ . . . Si r 
with coefficient 

f _ ^ \ [r/2] +nr+h H hir 

Applying T* and using ( 2.10|) we see that the monomial s n -i r s r 



-i r J n—i r —± • • 



pears in T*(j(a r )) with coefficient 

/•jNw+iiH Mr 

and in 7(0",.) with coefficient 

(_1 ^[r/2]+uH h«r 

This shows that T*(a r ) = (-l)[ r / 2 l +n V r . □ 

3. Calculation of equivariant cohomology 

Our purpose in this section is to compute the cohomology of G n /Z 2 , the factor- 
space of the space of closed string configurations G n = G(S m ; A, A, n) with respect 
to Z2-action given by the reflection T : G n — > G n . For n even T acts freely, and 
if*(G n /Z 2 ; Z) coincides with the equivariant cohomology of G n . 

The problem is trivial for m = 1; therefore everywhere in this section we will 
assume that m > 1. 
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To compute the equivariant cohomology we will apply the method of the Morse 
theory. Namely, we will consider the simplest billiard in the standard unit sphere 
S m C R m+1 and the function of negative total length 

n 

L:G n = G(S m ;A,A,n)^R, (x u . . . , x n ) i-> - \ x i ~ x i+i\- ( 3A ) 

i=0 

Here we understand Xq = x n+ \ = A. The critical points of L are the billiard 
trajectories in S m , which start and end at A and make n reflections. All such 
trajectories can easily be described. 

Namely, fix a vector a G S' m , a J- A, orthogonal to A and an angle 

^k = —r, fc = l,2,...,[(n+l)/2]. (3.2) 
n + 1 

This choice (a,ipk) determines the following billiard trajectory (x±, . . . ,x n ), where 

Xj = A cos(jip k ) + a sin(j^ fc ), j = 1, . . . , n. 

Note that for n odd the trajectory determined by the pair (a, ip( n+ i)/2) does not 
depend on a; it has the form (x\, . . . , x n ), where xj = A for j even and Xj = —A 
for j odd. 

We will denote by 

V p cG n , p = 0, 1, . . . , [(n - l)/2] 
the variety of trajectories determined by all pairs (a, ip^), where 

k = [(n+ l)/2] -p (3.3) 

and a _L A is an arbitrary point of the sphere S m ~ l C S m orthogonal to A. 

If n is even then every sub manifold V p is diffeomorphic to sphere 5 m_1 . 

If n is odd, then V is a single point and Vi, . . . , V[(„_i)/ 2 ] are diffeomorphic to 
the sphere S' m_1 . 

The following statement is similar to Proposition 3.1 of I.K. Babenko 

Proposition 4. Each V p C G n is a nondegenerate critical submanifold of function 
L in the sense of Bott. 

If n is even then index of each V p equals 2p(m — 1) forp = 0, 1, . . . , (n — 2)/2. 

If n is odd, then index of Vq equals and for p = 1, . . . , {n — l)/2 the index of 
V p equals (2p — l)(m — 1). 

Proof. Let e%, . . . , e m+ i G R m+1 be an orthonormal base. We may assume that 
A — e\. We want to calculate the Hessian of function L at a billiard trajectory 
c fc = (xi, . . . , x n ) G G n , where 

= cos(V> fc )ei + sin(-0fc)e 2 , j = 1, . . . , n, 

and 

2nk 

^ fc = ^I' = !,...,[(« + l)/2]. 
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Let xj- denote the orthogonal vector to Xj lying in the (ei, e 2 )-plane, i.e. 

xf = cos(tp k + 7r/2)ei + sin(*0 fe + 7r/2)e 2 . 

Any tangent vector Y G T Ck G n = Q)jT Xj S m is determined by numbers fj, r j G R, 
where r — 0, 1, . . . , m — 1 and j — 1, . . . ,n, such that the component of F in T Xj S m 
equals 

m— 1 
r=l 

A direct calculation of Hessian H(L) Ck (Y,Y) of L gives the following quadratic 
form in variables fj, r j: 

1 n 

//(L) Cfc (y,y) = -sin^^.^^-^+i)^ 

j=0 

(3.4) 

m—1 

^ v Q^fc(/^r,l) • • • ) /^r,n) j 
r=l 

where in the first sum we understand /xo,o = = /^o,n+i and in the second sum the 
symbol Q^(yi, ■ ■ ■ , y n ) denotes the following quadratic form 

n n—l 

Qi,(yi,...,y n ) = -2cos(^) • ^yf + 2 ^y^+i. 

i=l i=l 

We see that the Hessian splits as a direct sum of m quadratic forms corresponding 
to different values r = 0, 1, ... ,m — 1. The terms involving /i 0i j (the first sum) 
give a positive definite quadratic form. The rest m — 1 form are identical and their 
index and nullity equal the index and nullity of Q^, k . Hence we conclude that the 
index and nullity of the Hessian equals m—1 times the index and nullity of the 
form . 

In order to calculate the index of Q^ k we observe that the eigenvalues of the 
following symmetric n x n-matrix 






1 


.. 


. 





1 





1 . . 


. 
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.. 


. 
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. 


1 








.. 


. 1 






are given by 

\ s = 2 cos I , s — 1, 2, . . . , n (3.5) 



+ (2sin(^/2))- 1 - 
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and the eigenvector (t> 1)S 



V 3,s 



sin 



) corresponding to X s is given by 

TCjs 



n + 1 



J 



n. 



(3.6) 



This claim can be checked directly. 
Therefore the eigenvalues of Q^ k are 



cos 



ITS 



n + 1 



— cos 



2nk 
n + 1 



1,2, 



n 



(3.7) 



and the eigenvectors of Q^ k are given by 

Hence the index of Q^ k equals the number of integers s, such that 2k < s < n, 
which is n — 2k for 2k < n and if k — (n + l)/2 and n is odd. Since (according 
to (|3.3| )), k = \{n + l)/2] — p, we conclude that index of Q^ k equals 



n - 2k = n - 2([(n + 1) /2] - p) 



2p, 
2p- 



1. 



if n is even 
if n is odd. 



The special case k = (n + l)/2 for n odd corresponds to p = 0; in this case the 
index and nullity of Q^ k equal 0. 

From ( p.7| ) we see that the nullity of Q^ k equals 1 for any k unless n is odd and 
k = (n + l)/2. 

The discussion above proves that on any critical submanifold V p the dimension 
of the kernel of Hessian of L equals the dimension of V p ; hence all submanifolds V p 
are nondegenerate in the sense of Bott and their indices are as stated. □ 

The normal bundle v(y p ) splits as a direct sum v+{Vp) © v_(V p ) of the positive 
and negative normal bundles with respect to the Hessian of L. One may describe 
the negative normal bundle i>_(V p ) as follows. 

Lemma 5. The negative normal bundle u^(V p ) to V p is 

'£©£©•••©£, if n is even, 



2p 



■v 

times 



£, if n is odd and p > 0, 



(3. 



2p-i times 



m—l 



where £ denotes the tangent bundle of sphere S 

Proof. Let S" 11-1 c S m be the equatorial sphere consisting of unit vectors orthogo- 
nal to A. Any point a G S" 1 " 1 and an angle (3^) determine a critical submanifold 
V p . Fix an eigenvalue \ s (given by ( |3.5|) ), such that expression (|3.7|) is negative. 
Consider the subbundle u s (V p ) of the normal bundle v{y p ) consisting of eigen vec- 
tors of the Hessian with eigenvalue A s . We want to show that v s (y p ) is isomorphic 
to £. This would clearly imply the Lemma. 
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Consider a billiard trajectory Ck = (x±, . . . , x n ) G G n in the plane of vectors a 
and A, where 

Xj = cos(ip k )A + sin(^ fc )a, j = 1, . . . , n, 

and 

^ = ^' k=[(n + l)/2]-p. 

Denote by £ a the (m — l)-dimensional subspace orthogonal to a and A. We will 
show that there is an isomorphism between the fiber of VsiYp) over Ck and £ a , which 
depends continuously on a. 

Let Vj G T x .S m , where j — 1, 2, . . . , n, be a sequence of tangent vectors. Using 
( |3~^] ) and ( |3.6| ) we find that a sequence of vectors (t>i, . . . , v n ) belongs to the fiber 
of v s {V p ) over the configuration c& = (xx, . . . , x n ) if and only if 



sin 



TCJS 



G Co, and vj = 7 r- • v 1: j = 1, . . . , n. (3.9) 

7TS 



sin 



n + 1 



We see that the first vector vi uniquely determines a tangent vector (t>i, . . . , i> n ) 
to a configuration Ck in the eigen-deriction A s . Moreover, v\ can be an arbitrary 
vector in £ a . □ 

Since £ is orientable we obtain: 

Corollary 6. The negative normal bundle v_{V p ) is orientable. 

Note that this Corollary is trivial for m > 2 since then the sphere S 1 " 1-1 is simply 
connected. 



Corollary 7. The function L : G n — ► R (^c/. ( j&jj ) is a perfect Bott function. 
Proof. Note that the critical value L(V P ) equals 

(Ink \ 
-^yj, where k = [(n + l)/2] - p. 

Hence for p < p' we have £(Vp) < L(V^/). 

Choose constants c , ci, . . . , C[( n _i)/ 2 ] G R such that 

L(Vp) < c p < L(Vp+i), p = 0,l,...,[(n-3)/2], and L(V [{n ^ 1)/2 ]) < C[(n-i)/2]- 

Each F p = oo,c p ]) C G„ is a compact manifold with boundary and we 

obtain a filtration 

F C F\ C • • • C F[( n _i)/ 2 ], 
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and the inclusion F^ n _iy 2 ] —> G n is a homotopy equivalence (as follows easily from 
Proposition 4 in ||). Using Corollary ^| and the Thorn isomorphism we obtain 

W(F P ,F P ^;Z) ~ W- ind ^(V p ;Z) = 

Z, if j = md(V p ) or j = md(V p ) +m-l (3.10) 
0, otherwise 

This holds true also for p = if we understand F-i = 0. 

Suppose that n is even. Then cohomology group W(F p ,F p ^x;Z) is isomorphic 
to Z for j = 2p(m — 1) and for j = (2p + l)(m — 1) and vanishes for all other j. 
Comparing with additive structure of H*(G n ; Z) given by Theorem |3] we find 

[(n-l)/2] 

H*(G n ;Z)~ H*(F P ,F P ^;Z), (3.11) 

p=0 

which means perfectness of L. 

Suppose now that n is odd. Then H^Fq, F_i, Z) is Z for j = and vanishes for 
all other values of j. If p > then 

Z, for j = (2p — l)(m — 1) or j = 2p(m — 1) 



HHF F ,-Z) ~ I Z ' for ^' = ( 2 
1 p ' p- 1 ' J ~ \ 0, otherwise 



and thus perfectness ( |3.11| ) also holds. 

Alternatively, for m > 2 perfectness (|3.11|) follows without using Theorem R| by 



considering the spectral sequence of filtration F p 

E™ = H?+«{F p , F p _ r , Z) => H p+q (G n ; Z) 

and observing that for any of its differential d r , with r > 1, either the source or 
the target vanish. Therefore, E\ = E^. Moreover, every diagonal p + q = c of 
contains at most one nonzero group. If m = 2 the differential d\ has nonzero 
source and target, and so the above argument does not work. □ 

From this point we will assume that n is even. 

Then the reflection T : G n — > G n acts freely and our purpose will be to calcu- 
late the cohomology of the factor-space G' n = G n /Z 2 . Function (|3.1| ) is reflection 
invariant and so it determines a smooth function 

L' : G' n — > R,. 

The critical points of L' form nondegenerate (in the sense of Bott) critical sub- 
manifolds 

K)J ^1) • • • ) K/2-H 

where V p = V p /Z 2 . Index of V p equals 2p{m — 1) (as follows from Proposition 
£|). Since each V p can be identified with S m ~~ l and under this identification the 
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reflection T acts as the usual antipodal map, we see that each V p ' is diffeomorphic 
to the projective space RP m_1 . 

Corollary 8. The Poincare polynomial of G' n = G(S m ; A, A, n)/Z 2 with coeffi- 
cients in field Z 2 is 

jra ^n(m— 1) j 

t- 1 ' t2(m-l) _ ! ' 

and i/ie sitm of Betti numbers with coefficients in Z 2 mn/2. 

Proof. We will give here a simple proof working for m > 2. The case m = 2 will 



follow from Theorem |TT| below. 

Consider filtration F C Fi C • • • C F n / 2 _i C G n as in the proof of Corollary 0. 
Let denote F p /Z 2 . We obtain a filtration Fq C F{ (Z • • • C i 7 ^ /2 _ 1 C such 
that the inclusion i^, 2 _ 1 C GJ, is a homotopy equivalence and 

^■(F;, F' p _ x - Z 2 ) ~ ff- 2 K-D( RP ™-i ; z 2 ), p = 0, 1, . . . , n/2 - 1 

(using the Thorn isomorphism). Hence iF(F', Fp_ x \ Z 2 ) is nonzero (and one- 
dimensional) only for 2p(m — 1) < j < (2p + l)(m — 1). The spectral sequence of 
filtration F^ 

E p, q = h ^(f;, f;_ i; z 2 ) => f^(g; ; z 2 ) 

has F^' 9 ~ Z 2 for p(2m — 3) < g < p(2m — 3) + (m — 1) and Ff' 9 = otherwise. 
Hence for any differential rf r , where r > 1, either the source or the target vanish. 
Therefore, E\ = and our statement follows. □ 

We will now calculate the Stiefel- Whitney classes of the negative normal bundle 
v_(Vp). In particular, we will find out for which p this bundle is orientable. This 
information is needed for computing the integral cohomology of G' n . 

Lemma 9. The total Stiefel - Whitney class of the negative normal bundle v-(Vp) 
equals 

(l+a)Km-i) e H*(V;;Z 2 ), 
where a G F 1 (VC / ; Z 2 ) ~ Z 2 denotes the generator. 

Proof. As in the proof of Lemma |5| we obtain that the negative normal bundle 
u_(Vp) splits as a direct sum of 2p vector bundles rj s of rank m — 1, one for each 
negative eigenvalue 

its \ ( 2irk 

cos I — cos 



n + 1 / \n + 1 



A s = 

of the Hessian. Here k = n/2 — p. 

Let r denote the tangent bundle of RP m ~ . Let 7 -1 be a rank m—1 vector 
bundle over RP m_1 such that its fiber over a line £ G RP m 1 is the orthogonal 
complement £ . 
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We claim that 

^ J r if s is even, 
113 ~ \ 7 1 - if s is odd. 

Indeed, this bundle is obtained from the tangent bundle £ of S 1 " 1-1 (cf. Lemma (5|) 
by identifying the antipodal points, and under this identification the first vector 
v\ should be replaced by the last vector v n (cf. (|3.9| )). Formulae fl3.9|) show that 

V n = — COs(7Ts) • V\ = ( — • V\ 

and hence the bundle r\ s is obtained from £ by identifying the fibers over points a 
and —a with a twist (— l) s+1 . This implies our claim, cf. |11 . 



For given p there are equal number of negative eigenvalues A s of the Hessian 
on V p with even and with odd s. Therefore the bundle v_(V') is isomorphic to a 
direct sum of p copies of r © 7 . 

The total Stiefel - Whitney class of 7 -1 is (1 + a)" 1 and the total Stiefel - 
Whitney class of r is (1 + a) m (cf. |pT|). Hence the total Stiefel - Whitney class 
of the negative bundle is 

[(1 + a )-l . (1 + a )m]P = (1 + a) (m- Dp. 

□ 



Corollary 10. If m is odd, then the negative normal bundle v_(y£) is orientable 
for any p. 

If m is even, then the negative normal bundle z/_(V^') is orientable for all even 
p and it is non- orientable for all odd p. 

Proof. By Lemma ^ the first Stiefel - Whitney class of U-(V') is p{m — l)a. This 
implies our statement. □ 

Recall our permanent assumption m > 1 and n is even. 

Theorem 11. If m > 1 is odd then 



W(G> n , Z) 



If m is even then 



Z, for j — 2i(m — 1), where i — 0, 1, . . . , n/2 — 1, 

Z2 ; for j even satisfying 2i(m — 1) < j < (2i + l)(m — 1) 

with i as above, 
otherwise. 



H\G' n ,Z)~{ 



Z, for j = (4r + e)(m - 1), r = 0, 1, . . . , [{n - 2)/4], e = 0, 1, 
Z2, /or j = Ar(m — 1) +i, or j = (4r' + 2)(m — 1) + z' ; where 

i — 2, 4, . . . , to — 2, r is as above, i' = 1, 3, . . . , to — 1 and 

< r' < (n-4)/4 ; 
0, otherwise. 
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Proof. Consider filtration Fq C F[ C • • - C F' n , 2 _ x C (cf. proof of Corollary ||) 
and the associated spectral sequence 

e ,p,<i = H p+i^ F > p i] Z ) HP + i(G' n , Z). 

Fp — F'^ contains a single critical submanifold V' ~ RP" 1 " 1 with index 2p(m — 1). 
The normal bundle to VC' is orientable if pirn — 1) is even and it is non-orient able 
if pirn — 1) is odd. The Thorn isomorphism gives 

( H j - 2p ( m -V(KP m - 1 ; Z), if p(m - 1) is even, 
W{F' p ,F' p ^Z)^\ (3.12) 

[ iji-2p(m-l)( R pm-l. ±z ^ if p ( m _ y ig odd _ 

Here ±Z denotes the nontrivial local system of groups Z over RP m_1 ; its mon- 
odromy along the generator of ^(RP" 1-1 ) is multiplication by —1. 
For m even we have 

Z for j = and j = m — 1 , 
#J(RP"»-i;Z) = <j Z 2 for j = 2, 4, . . . , m — 2, 
otherwise 



and 



otherwise 



For m odd we have 

( Z for j = 0, 
^(BP 1 " -1 ; Z) = ^ Z 2 for j = 2, 4, . . . , m - 1, 
I otherwise. 

Therefore, in the above spectral sequence holds E' v { q = for p(2m — 3) < q < 
p(2m — 3) + (m — 1). It implies that for m > 2, either the source or the target of 
any differential d T vanish. 

Hence for m > 2 holds E\ = E'oo and any diagonal p + q = const contains at 
most one nonzero group. This proves our statement for m > 2. 

Assume now that m = 2 and consider the first differential d\ : E'Y^ 1 ' 7 " — > E'Y ' . 
We have 

ET l,r „ H 2r-1 {F F Z) = ( Z, if r is Odd, 

1 v r i, r 2 , ; | Z 2 , if r is even 

and 

E'Y ~ # 2r (F r , F r _ l5 Z) = ( °' I! r is ° dd ' 

1 v 7 I Z, it r is even. 

We see that d\ vanishes since there are no nonzero homomorphisms E' r {~ ,r — > E'Y 
for any r. 
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The higher differentials d r , r > 2 vanish by obvious reasons. Hence, the conclu- 
sion we made for m > 2, holds also for m = 2. □ 

The following Theorem is the main result of this section. It describes the mul- 
tiplicative structure of H*(G' n ; Z). Recall that we assume that n is even. 

Theorem 12. For m > 1 odd, H*{G' n ;Z) is the commutative ring given by se- 
quence of generators 

S . eH *^ 1 \G f n ;Z)~Z, < = 0,1,2,..., 

and 

eeH 2 (G' n ,Z)^Z 2 , 
satisfying the following relations 

(2i + 2j)\ . . 

OjOn = -, TT7 7T ' Oi+4, n /o = 0, Ot) = 1, 

J (2i)!(2j)! 3 1 
2e = 0, e (™+i)/2 = o. 

Ifm is even then H*(G' n ; Z) is the graded- commutative ring given by the generators 
5 z eH^ m - 1 \G' n ;Z)^Z, i = 0,1,2,..., 

and also 

e G H 2 (G' n ; Z) ~ Z 2 , a G fl™"^; Z) ~ Z, 6 G H 2m ^ 1 (G' n ; Z) ~ Z 2 , 
satisfying the following relations 

(2i + 2j)\ 

j = {2i)\{2j)\ ' +j) ^[("+ 2 )/ 4 ] = °' °° = lj 
2e = 0, e m l 2 = 0, 

a 2 = 0, ab = 0, ae = 0, 

2b = 0, 6 2 = 0, 

46 = (ifn = Ak + 2). 

Remark. For m = 2 the generator e disappears since one of the above relations 
reads e = 0. If m is even and n = 2 then 6 = since one of the relations gives 
5 b = 0. 

Proof. Consider the universal Z 2 -bundle S°° — > RP°° and the associated fibration 
S°°x Z2 G n — > RP°°, having G n as the fiber. The total space S°°Xz 2 G n is homotopy 
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equivalent to G' n . The Serre spectral sequence of this fibration converges to the 
cohomology algebra H*(G' n ; Z). The initial term is 



where TC q (G n ; Z), the cohomology of the fiber, is understood as a local system over 



From Theorem |3| we know that H q (G n ; Z) is either Z or trivial. There are two 
types of local systems with fiber Z over RP°°, which we will denote Z and ±Z. 
Their structure is determined by the monodromy along any noncontractible loop 
of RP°°, which is 1 in the case of Z and —1 in the case of ±Z. 

Q \ 

5(m-l) ..... 



4(m - 1) u 




3(m-l) 




m — 1 



iF(RP°°;ft 9 (G n ;Z)) 



RP 



oo 




1 



2 



m + 1 



m + 3 



m + 5 



V 



Figure 1 . Term E m of the spectral sequence for m odd 



Assume first that m > 1 is odd. From formula ( |2.6[ ) we find that 




for q = 2i(m — 1), 

for g = (2i + l)(m — 1), 
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where i — 0, 1, . . . , n/2 — 1. Hence we find 



F P,q 



Z, for p = and g = 2i(m — 1), 

Z 2 , if p > is even and q = Him — 1) 

or if p is odd and q = (2i + l)(m — 1), 
0, otherwise, 



where i — 0, 1, . . . , n/2 — 1. As a bigraded algebra, _E 2 can be identified with the 
tensor product 



E° 2 '* <g> £ 2 *'° <g> A, 



where 



E o,* ^ # 2 *(G n; Z), £ 2 *'° ~ iT(RP°°; Z), 



and A is an exterior algebra with A ' ~ Z, and A 1 '" 1-1 ~ Z 2 . If x G E\ ,m ~ l is the 
generator, then relation x 2 = follows from relation o\ = 2er 2 (in the notations of 
Theorem g). Here we denote H 2 *(G n \ Z) C H*(G n ; Z) the graded subring 

H 2 *(G n ,Z) = 0/f 2 ^- 1 )(G n ;Z). 

i 

The structure of the ring H 2 *(G n ; Z) follows from Theorem [| 

The first nontrivial differential is d m . Since we know the additive structure of 

H*(G' n , Z) (cf. Theorem P) we find that the differential d = d m : E^ m ~ x -> £ 2 m+1 '° 

must be an isomorphism. On the other hand (i : }^ 2%<<m x > _ > E™^ 2% ^ 

vanishes (since the range is the zero group). It follows that d : ^>^ m ~ 1 ^ — > 

^p+m,0'-i)(m-i) . g nonzero iff both p and j are odd. 

Figure [I] shows the nontrivial differential d = d m . The fat circles denote group 

Z and the small circles denote Z 2 . 

We conclude that the bigraded algebra E m+ i is isomorphic to the tensor product 

of algebras 

H 2 *(G n ,Z) <g>iT(RP m - 1 ;Z) > 

where H 2i( - m -V(G n ; Z) has bidegree (0, 2i(m-l)) and F^'(RP m_1 ; Z) has bidegree 
(2j, 0). It is clear that all further differentials vanish and hence E^ = E m+ i. Any 
diagonal p + q = c contains at most one nonzero group, and hence the algebra 
H*(G' n ; Z) coincides with E^. This proves our statement for m > 1 odd. 

Assume now that m is even. Recall that we always assume that n is even. From 
formula (|2.6|) we find that 

1), 

-1), 



<ui(r 7\ ~ / Z ' forg = 4i(m-l), or g= (4i + l)(m-l). 
1 n ' \ ±Z, for g = (4z + 2)(m - 1) or g = (4z + 3)(m 
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9(m- 1) 
8(to- 1) 
7(to- 1) 
6(m — 1) 
5(m — 1) 
4(to - 1) 
3(m- 1) 
2(m- 1) 

TO — 1 





ni 



to + 2 to + 4 



V 



Figure 2. Term of the spectral sequence for m even 

assuming that q < n(m — 1). Hence we find 

Z, for p = and q = (4z + e) (to — 1), where e = 0, 1, 
Z2, if p > is even and q = (4i + e) (to — 1) 



or if p is odd and q — (4i + 2 + e)(m — 1), 
0, otherwise. 



As a bigraded algebra, E 2 can be identified with the tensor product 



E 



.0,* 



E?°®B*'\ 



where 



E '* ~ 



H 4 *(G n ,Z)®C*, 



Ef ~ 



iT (RP°°;Z) 



C* is an exterior algebra with C° ~ Z and C m 1 ~ Z and 5*'* is an exterior 
bigraded algebra with B 0,0 ~ Z, and f? 1 '^" 1-1 ) ~ Z 2 . If y G i^' 2 ^ 1-1 ^ denotes the 
generator then y 2 = follows from relation cxf = 2o"4, cf. Theorem ^. We denote 
by H i *{G n \ Z) C H*(G n ; Z) the graded subring 

H 4 *(G n ;Z) = Q)H^ m - l \G n -Z). 
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Consider now the first nontrivial differential d = d m : E™ — > E^ rm,q m+1 . It is 
clear that it may be nonzero only for q of the form q = (2i + l)(m — 1). On the 



other hand, since we know the additive structure of the limit (cf. Theorem [II]), we 



conclude that d : E®'" 1 ' 1 = Z —>■ E™'° = Z 2 is onto. Using multiplicative properties 
of the spectral sequence we find that all the differentials shown on Figure ||] are 
epimorphic. In fact all differentials on Figure ^ except those which start at the 
q axis, are isomorphisms (since they act between isomorphic groups). As before, 
the fat circles denote Z and small circles denote Z 2 . 

Hence, to the next term E m+ \, survive classes a a = 5i, and also a = 2o~i, 
e G E^f +1 ~ Z 2 and b G i^+i and their products 5ja, 8^, and <5j6e J with 
j < m/2. It is clear that all further differentials vanish and in each diagonal 
p + q = c there is at most one nonzero group. Therefore we conclude that ring 
H*(G' n ; Z) is isomorphic to E m+ \. Its structure coincides with the description given 
in Theorem |I^. □ 

4. Equivariant Lusternik - Schnirelman theory via nonsmooth critical 

point theory 

In this section we firstly recall the basic notions of the critical point theory 
for nonsmooth functions, suggested recently in Q and f|. Secondly, we apply the 
nonsmooth critical point theory to get a simple independent exposition of a version 
of the equivariant Lusternik - Schnirelman theory of [0, 0, which we need for 
our applications to the billiard problems. One of the advantages of our approach 
is applicability to manifolds with boundary. 

Let X be a metric space endowed with the metric d. Given a point p G X and 
5 > 0, we denote by B(p, S) C X the ball of radius 5 centered at p. 

Definition 13. Let / : X — > R be a continuous function. The weak slope of f at 
a point p G X, denoted \df\(p), is defined as the supremum of all cr g [0, oo] such 
that there exists 5 > and a continuous deformation rj : B(p, 5) x [0, 5} — > X with 
the following properties 

d(r)(q,t),q)<t, f{r)(q,t))<f(q)-at 

for all q G B(p,5), t G [0,5]. 

A point p G X is said to be a critical point of function f if \df\(p) = 0. 



Example 14. Let X be a smooth Riemannian manifold without boundary, and 
let / : X — > R be a smooth function. Then the weak slope \df\(p) coincides with 
the norm of the differential ||d/(p)||, viewed as a bounded linear functional on the 
tangent space T P (X). 

Example 15. Let X be a smooth Riemannian manifold with boundary, and let 
/ : X — > R be a smooth function. A point on the boundary p G OX is a critical 
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point of / iff there is no tangent vector v G T p X pointing inside X such that the 
derivative v(f) < is negative. The last condition implies that 

df P \ Tp9 x = 0, (4.1) 

i.e. the gradient of / at point p G dX is orthogonal to the boundary dX. A 
point p G dX is a critical point of / iff ( |4.1|) holds and the gradient of / at p has 
the inward direction. It is clear that the above conditions are independent of the 
Riemannian metric. 



Proposition 16. Let f : X — > R be a continuous function on a compact metric 
space X. Then the number of critical points of f (in the sense of Definition [73 ) is 
at least c&t(X), the Lusternik - Schnirelman category of X. 

This follows from a much more general theorem 3.7 of 0. 
We will apply the nonsmooth critical point theory to equivariant critical point 
theory of smooth functions. Compare [HJ, [EJ. 

Proposition 17. Let M be a smooth compact G-manifold with boundary, where 
G is a finite group. Let f : M — > R be a G-invariant smooth function. Suppose 
that at points of the boundary p G dM the gradient of f does not vanish and has 
the outward direction. Then the number of G- orbits of points p G M with df p = 
is at least cat (M/G). 

Proof. Let X denote the space of orbits X = M/G. Function / determines a con- 
tinuous function / : X — > R. We want to show that any orbit x G X, representing 
points p G X with df p ^ 0, is not a critical point of / : X — > R in the sense of 



Definition 13. This would imply that the number of critical orbits of / is at least 



the number of critical points of /; the later can be estimated from below by cat(X) 



by Proposition 16 



We will assume that M is supplied with a G-invariant Riemannian metric. Let 
p G M be a point with df p ^ 0. We want to construct a smooth vector field v in a 
neighborhood of the orbit of p having the following properties: 

(a) v(f) p < 0; 

(b) the norm of vector v p equals 1; 

(c) v is G-invariant; 

(d) in case p belongs the boundary dM, the vector v p points inside M. 

To construct such vector field v one first finds a vector v p G T P M for each point p 
of the orbit so that (a), (b) and (d) are satisfied. It is then possible to extend the 
vectors v p to form a smooth vector field v in a neighborhood of the orbit of p with 
properties (a), (b), (d). Then (c) can be achieved by averaging. 

The flow determined by the vector field v, gives a deformation of a ball around 
the point {p} G X, which represents the orbit of p, showing that the slope \df\(p) 
is positive. □ 
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5. Proof of Theorem |] 

Let T C R m+1 be a compact strictly convex domain bounding a smooth hyper- 
surface X = dT. Given a point A e X and an integer n, consider the configuration 
space G n = G(X; A, A, n) (cf. ( |2.1| )) and the smooth function 

i=n 

L x '■ G n — > R, L x (xi, . . . , x n ) = — \xi — x i+ i\ 

i=0 

(the negative total length), where we understand x = A = x n+ i. This function is 
invariant with respect to reflection T : G n —> G n (cf. (|2.2|)). Hence Lx determines 
a continuous function 

L'x '■ G' n — > R, G' n = G n /T, (5.1) 

and the critical points of L' x (in the sense of Definition [13]) are in one-to-one 
correspondence with Z 2 -orbits of closed billiard trajectories in X, which start and 
end at A and make n reflections. This follows from Lemma 2 in M and from the 
argument in the proof of Proposition 

Note that for n is even, T acts freely on G n , the factor G' n = G n /T is a smooth 
manifold and the function L' x is smooth. In this case the nonsmooth critical point 
theory coincides with the usual one. 

We claim: 

The number of critical points of L' x is at least the Lusternik - Schnirelman 
category c&t(G' n ); moreover, assuming thatn is even and the function L' x is Morse, 
the number of critical points of ( \5. J\ ) is at least the sum of Betti numbers of G' n 
with Z 2 coefficients. 

The italiced statement does not follow directly from the traditional Morse- 
Lusternik-Schnirelman theory, since G' n is not compact. However, as in 0, [|J, 
one may fix e > and consider the following compact subset 

n 

G e C G n , G e = {(x 1 ,...,x n ) E X xn :Y[\ Xi -x i+ i\ > e}. 

i=0 

If e > is small enough then: 

(a) G e is a compact manifold with boundary; 

(b) the inclusion G e C G n is a Z 2 -equivariant homotopy equivalence; 

(c) all critical points of L x are contained in G E ; 

(d) at every point of dG e the gradient of L x has the outward direction. 
Cf. 0, Proposition 4.1 and 0, Proposition 4. 

Since G' e = G e /T is a compact smooth manifold with boundary, we may apply 
the Morse-Lusternik-Schnirelman theory to it. Condition (d) implies that the 
critical points of the restriction of L' x on dG' e should not be taken into the account 
(cf. Proposition |T7|) . Therefore, the number of critical points of L' x \G> t is at least 
cat(Gg) = cat(Gr^). If L' X \ G ^ is Morse, then the number of its critical points is 
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at least the sum of Betti numbers of G' e , which is the same as the sum of Betti 
numbers of G' n . 

In the proof of statement (I) of Theorem [1] we will use the following general 
simple remark: 

for any regular covering map p : X — » X holds 

cat(X) > cat(X). (5.2) 

Indeed, if A C X is an open subset which is contractible to a point in X then 
A = p~ l (A) is a disjoint union of open subsets of X, such that each is contractible 
to a point in X. Hence, any categorical open cover A\ U A 2 U ■ ■ • UA^ = X produces 
a categorical open cover Ai U A 2 U • • ■ U A k of X. 

The above remark applies to the two-fold cover G n — > G' n giving 

cat(G'J > cat(G n ) > cl(G n ) + 1, 

where cl(G n ) is the cohomological cup- length of G n (Froloff - Elsholz Theorem). 

Now we use Theorem |3| to compute the cup- length of G n . In m > 3 is odd then 
al~ l = (n-l)!o- n _i ^ G H^ 1 ^-^ (G n ; Z) and hence cl(G n ) = n-1. Therefore 
cai(G' n ) > n. This proves statement (I) in the case m > 3 odd. 

If m is even, then the longest nontrivial cup-product in H*(G n ; Z) is 

o\o\~ x if n = 2k is even, 
of, if n = 2k + 1 is odd. 

We conclude that for m even the cup-length of G n equals [n/2] and therefore 
caA(G' n ) > [n/2] + 1. Together with the information collected above this proves 
statement (I) for m even. 

To prove statement (II) of Theorem [I] we will use Theorem [12] to estimate the 



Lusternik - Schnirelman category of G' n . Note that Theorem [12] requires the as- 
sumption that n is even. Suppose first that m > 3 is odd. Then (in the notations 
of Theorem IT3) we have a nonzero cohomology product 

W 22 ...cW m - 1)/2 , (5.3) 

where s is the largest integer with 2 S+1 — 1 < n/2 — 1, i.e. s = [log 2 (n)] — 2. 
Note that class e G H 2 (G' n ; Z) has order 2, i.e. 2e = 0. Nontriviality of the above 
product is equivalent to the claim that the product S1S2S22 . . . ^ is an odd multiple 
of the class 5i+2+2 2 + - +2 s - Indeed, we use the relation 

x , ( 2i + 2j \ , 
Oidj = [ 2i ) 

and the well-known fact that the binomial coefficient I „. ) is even if and 



2% 

only if % and j, in their binary expansions, have a 1 on the same place. 

Now we will use the notion of category weight of a cohomology class, introduced 
by E. Fadell and S. Huseini M. We claim that the category weight of e G H 2 (G' n ; Z) 
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equals 2. Indeed, e equals the image of a class e' G Z 2 ) under the Bockstein 

homomorphism (3 : H l (G' n \7i2) — > H 2 (G' n ;Z), i.e. e = /3(e'). Hence by Theorem 
(1.2) of E. Fadell and S. Huseini 0, the category weight of e is 2. Therefore, 
nontriviality of product (|5.3| ) implies 

Tfl — 1 

cat(G'J > (s + 1) + 2 ■ — — + 1 = [log 2 n) + m - 1. 

This proves statement (II) for m > 3 odd. 

Consider now the case when m > 2 is even. First we will assume that n > 8 
and n + 2 is not a power of 2. Then we have a nontrivial cohomological product 

6 1 6 2 5 2 2...6 2 sbe (m - 2)/2 , (5.4) 

"n + 2~" 



where s 



log 2 



4 



1. As above, nontriviality of 



implies 



cat(G' n ) > (s + 1) + 1 + 2 



m — 2 



log 2 



n 



+ m > [log 2 n] + m — 2. 



If n + 2 = 2 r is a power of 2, where r > 4, then the product 

5 1( 5 2 5 2 2...5 2s e (m - 2)/2 , 

(we skip b because of the last relation in Theorem |12|) is nonzero, where s = r — 3. 
In this case we obtain 

cat(G^) > (a + 1) + 2 hl=s + m= [log 2 n] + m - 2. 

We are left to consider the cases n = 2, 4, 6 with m even. Here we have a 
nontrivial cup-product be^ m ~ 2 ^ 2 and hence 

rn — 2 

cat(G^) > 1 + 2 • + 1 = m. 

This implies the estimate of statement (II) of Theorem [1] for the specified values 
of n and m. 

For m > 1 statement (III) of Theorem |T| follows from Corollary [5[ If m = 1 then 
the space G[ = G^S 1 ; A, A, n)/Z 2 consists of n/2 connected components and each 
is contractible; this can be established by arguments similar to those used in §7 of 
|§. Hence the sum of Betti numbers of G[ is n/2. 

□ 



6. Cohomology of cyclic configuration spaces of spheres 

The cyclic configuration space G(X,n) of a space X is defined (cf. 0) as the 
set of all configurations (xi, . . . , x n ) G X xn with Xi ^ Xi + i for i = 1, . . . , n — 1 and 
x n ^ x\. The dihedral group D n acts naturally on G(X,n). 

In we showed that information about the cohomology ring of the factor- 
space G(S m , n)/D n leads to estimates on the number of n-priodic orbits of convex 
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billiards in (m + l)-dimensional space R m+1 . The rings H*(G(S m , n); Z 2 ) and 
H*(G{S m ,n)/D n ;Z 2 ) were computed in [§. 

In this section we will describe the cohomology of the cyclic configuration space 
G(S m , n) with other fields of coefficients. It turns out that the answer depends on 
the parity of m; therefore we state the even and the odd dimensional cases in the 
form of two separate theorems. 

The results of this section will be used in the proof of Theorem ^|. 

Theorem 18. Let m > 3 be odd. The ring H*(G(S m , n); Q) is given by generators 
u G H m (G(S m ,n); Q), a, L G H i( - m - 1 \G{S m , n); Q), i = 1, . . . ,n - 2, 



and relations: 



(i + j) 



a i+j , if i+j<n-2, 
u 2 = 0, (JiOj = { V -' J[ (6.1) 

0, if i + j > n - 2, 

One may show that the statement of Theorem [T^ holds with Q replaced by an 
arbitrary field of coefficients k. However, the short proof we give below, works 
only in the case k = Q. On the other hand for our purposes of this paper it is 
enough to know the rational cohomology H*(G(S m ,n); Q). The case of a field k 
of positive characteristic may be proven using Theorem 3 of and computing the 
spectral sequence similarly to the proof of Theorem 4 in 0. 

The following theorem gives the answer for m even. 

Theorem 19. Let k be a field of characteristic ^ 2. For any even m > 2 and odd 
n>3 the cohomology algebra H*(G(S m ,n);k) has generators 

w e H 2m -\G{S m ,ny,k), a 2i e ^""^(^(S" 1 , n); k), % = 1, . . . , (n - 3)/2, 

such that 

(?2(i+j), ifi+j<(n-3)/2. 
w 2 = 0, a 2i a 2j = { v --3' (6.2) 



Proof of Theorem IT! 




i + j > (n-3)/2. 



(6.3) 

where the image of a cyclic configuration (xi, . . . ,x n ) G G(S m ,n) under projec- 
tion p is given by p(x±, . . . ,x n ) = X\. The fiber of p is the configuration space 
G(S m ; A, A, n — 1). Consider the Serre's spectral sequence of this fibration. The 
cohomology of the fiber G(S m ; A,A,n — 1) is described by Theorem it has gen- 
erators ci, ... , 0„_2, which multiply according to ( |2.3| ). 
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Figure 3. Continuous family of configurations 

This spectral sequence may have only one nonzero differential d m . We will show 
that this differential vanishes d m — 0. This would clearly imply our statement. 

Since we may write Oi = it is enough to show that d m (<7i) = 0. 

Vanishing d m (<7i) = follows from the fact that fibration ( |6.3| ) admits a continuous 
section s : S m —>■ G(S m ,n) and thus the transgression is trivial. To construct s, 
fix a nowhere zero tangent vector field V on the sphere S m (recall that m is odd). 
For x G S m , the tangent vector V(x) determines a half circle starting at x, tangent 
to V(x) and ending at the antipodal point —x. Then the section s can be defined 
by 

s(x) = (xi,x 2 , ■ • • ,x n ), xeS m , 

where and the points x%, ■ ■ ■ , x n -\ are situated on the half circle 

making equal angles as shown on the following picture. 
Analytically we may write 

Xj = cos ) x + sin ) V (x), j = 1, . . . , n. 

\ n — 1 / \ n — 1 J 

□ 

Proof of Theorem [Tj^. First we will assume that m > 2; the case m = 2 will be 
treated separately later. 

We will describe the additive structure of H*(G(S m , n); k), using approach of 
the Morse theory. 

Let 5* m C R m+1 be the unit sphere. Consider the total length function 

L : G(S m ,n) -» R, 
where for (xi, . . . , x n ) G G(S m , n) we have 

L(xi,x 2 , ■ • . ,x n ) = -\xi - x 2 \ - \x 2 -x 3 \ \x n - x x \. 
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The critical points of L are n-periodic billiard trajectories in the unit sphere; 
hence the critical configurations are regular n-gons lying in two-dimensional central 
sections of the sphere. A regular n-gon is determined by two first vectors Xx,x% G 
S m , which must make an angle of the form 

2n n — 1 . 

a p = (— p), where p = 0, 1, . . . , (n - 3)2. 

n 2 

Recall, that we assume that n is odd. Fixing p — 0, 1, . . . , (n — 3)/2, we obtain 
a variety of critical configurations, which we will denote by V p C G(S m ,n). Each 
V p has dimension 2m — 1 and is diffeomorphic to the Stiefel manifold of pairs of 
mutually orthogonal vectors in R m+1 . Since we assume that m is even and the 
characteristic of k is ^ 2, we have H*(V P ; k) ~ H*(S 2m ~ 1 ] k). Note also that V p is 
simply connected (since m> 2). 

I.K. Babenko has shown (cf. 0, Proposition 3.1) that function L is nondegen- 
erate in the sense of Bott and the index of each critical submanifold V p equals 
2p{m — 1). Moreover, it is clear that L(V P ) < L(V p i) for p < p' . 

Fix e > small enough and consider the submanifold G e (S m ,n) C G(S m ,n), 
where 

n 

G e (S m , n) = {( Xl , ...,x n )e (S m ) xn Hlxi- x i+1 \ > s}. 

i=l 

If e > is sufficiently small then (according to Proposition 4.1 of 0) G e (S m ,n) is 
a compact manifold with boundary containing all the critical points of L and such 
that the inclusion G £ (S m ,n) C G(S m ,n) is a _D n -equivariant homotopy equiva- 
lence. Moreover, at every point of the boundary dG e (S m ,n) the gradient of L has 
the outward direction. 

Choose constants Co, c\, . . . , C( n _3)/2 such that L(V P ) < c p < L(V p+ i) for < p < 
{n — 3)/2 and C( n _3)/2 = 0. Let 

F p = L-\(-oo,c p ])nG £ (S m ,n). 

We obtain a filtration F C -F\ C ■ ■ ■ C F^ 3 y 2 = G £ (S m , n). Since the inclusion 
F(n-S)/2 C G(S m ,n) is a homotopy equivalence, we may use the spectral sequence 
of this filtration in order to calculate the cohomology of G(S m ,n). 

We claim that this filtration is perfect, i.e. the Poincare polynomial of the cyclic 
configuration space G(S m , n) equals the sum of the Poincare polynomials of the 
pairs (F p , The initial term of the spectral sequence is 

E{' q = H^{F P) F p ^). 

Uisng the Thorn isomorphism (recall that V p is simply connected), we find that 
£P(F p ,Fp_i;k) is isomorphic to H j - % ^ m - l \V pi k); hence H j (F p , k) is one- 
dimensional for j = 2p(m — 1) and for j = 2p(m — 1) + 2m — 1 and vanishes for all 
other values of j. This follows since in — F p there is a single non-degenerate 
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critical submanifold V p , which has index 2p(m — 1) and V p is diffeomorphic to the 
Stiefel manifold V m +\,2- 

The gradient of L at points of the boundary dG £ (S m ,n) has the outward direc- 
tion and hence the points of the boundary do not contribute to the usual statements 
of the Morse - Bott critical point theory. 

For a given p there are precisely two values of q such that Ef ,q is nonzero (q = 
p(2m — 3) and q = p(2m — 3) + 2m — 1). From the geometry of the differentials 
we see that all the differentials d r , r > 1, must vanish if m > 2. This proves 
that the cohomology H^(G(S m , n); k) is one dimensional for j = 2p(m — 1) and 
j = 2p(m— l)+2m — 1, where p = 0, 1, . . . , (n— 3)/2, and Hi(G(S m , n); k) vanishes 
for other values of j. 

Having recovered the additive structure of H*(G(S m ,n); k), we may use Theo- 
rem |3] to find its multiplicative structure. The mapping (xi,x 2 , ■ ■ ■ , x n ) i— >■ Xi is a 
Serre fibration G(S m , n) — > S 1 " 1 ; its fiber is G(S m : A, A, n — 1). The Serre spectral 
sequence has only two nonzero columns and d m is the only differential which could 
be nonzero. In the 0-th columns we have classes cr, in dimensions i{m — 1), and in 
the m-th column we have classes Oiix having dimension i(m — 1) +m, cf. Theorem 
H Since we already know the additive structure of H*(G(S m , n); k), we conclude 
that the differential 

, . p 0,i(m-l) pm,(i-l)(m-l) 
u m • - c '2 2 

is an isomorphism for i odd and vanishes for i even. Hence the classes 02i and 
cr 2 j + iM survive. Now, we set w = aiu, and the conclude that H*(G(S m ,n); k) has 



the multiplicative structure as stated in Theorem 19 




P 



Figure 4. Nonzero terms of the spectral sequence for m = 2 
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For m = 2 the above argument, based on the spectral sequence of the filtration 
F C Fi C • • • C i 7 '(n-3)/2 = Ge(S m ,n), is not sufficient, since, in principle, this 
spectral sequence could have a nonzero differential, shown on the picture. Also, for 
m = 2 the critical submanifolds V p are not simply connected and so the Thorn iso- 
morphisms for the negative normal bundles of the Hessian may require additional 
twists by flat line bundles (depending on the orientability of the negative normal 
bundles of the critical submanifolds). 

However, in the case m = 2 a different argument can be applied. Consider the 
action of SO (3) on G(S 2 ,n) arising from the standard action of SO (3) on S 2 . Fix 
a point A G S 2 and consider G(S 2 ; A,A,n— 1) as being canonically embedded in 
G(S 2 ,n). We obtain the map 

SO(3) x G(S 2 ;A,A,n- 1) -> G(S 2 ,n), {R,c) i-> Rc, (6.4) 

given by applying an orthogonal matrix R G SO (3) to a configuration of points on 
the sphere c G G(S 2 ; A, A, n — 1). It is easy to see that ( |6.4j ) is a fibration with fiber 
S 1 . If c = (A,x±, . . . ,x n -i) is a configuration of points on S 2 such that A ^ x\, 
Xi 7^ for i = 1, . . . , n — 1 and x n _i 7^ A, then the fiber of fibration ( |6.4j ) over 
c consists of the space of all pairs (R-^, R^c)), where R$ G 5*0(3) denotes the 
rotation by angle G [0, 27r] about A. 

The cohomology algebra of the total space of this fibration 

H*(SO(3) x G(S 2 ; A,A,n — l);k) ~ iT(S 3 ; k) ® iJ*(G(5 2 ; A, A, n - 1); k) 

is given by Theorem |[ It has a generator w, with degw = 3 (coming from a 
generator of H 3 (SO(3); k)) and also classes <jj, where i = 0, l,...,n — 2, with 
deg <7j = z, which are pullbacks of the generators of H*(G(S 2 ; A, A, n — l);k), cf. 
Theorem ||. We have the relation w 2 = and each product a. t aj equals a multiple 
of <7 i+ j, the coefficient indicated in formula (|2.4|) . 

Let us show that the restriction map from the total space to the fiber 

H\SO(3) x G(S 2 ; A, A, n - 1); k) -> tf 1 ^ 1 ; k) 

zs onto. Since if 1 (S'0(3); k) = 0, our statement is equivalent to the following. 
Let c = (A,xi, . . . , SE n _i) be a fixed configuration. We obtain an embedding / : 
S 1 -> G(S' 2 ; A, A, n - 1) given by ^ -R^(c), where G [0, 27r]. We claim that the 
induced map /* : if 1 (G r (5' 2 ; A, A, n — 1); k) — > H 1 ^ 1 ] k) is onto. In other words, 
we want to show that the cohomology class f*{o~i) G H 1 (S l ; k) is nonzero. 

We may assume that the antipode A' of A does not appear in the configuration 
c. Identify S 2 — A' with R 2 using the stereographic projection with A' as a center; 
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this leads to the following commutative diagram 

S l G(R 2 ; 0, 0, n - 1) 

\ / ih 

G{S 2 ;A,A,n-l) 

where g is given by rotations of a fixed configuration d = (0,yi,yz, ■ ■ ■ ,y n -i) of 
points on the plane, d G G(R 2 ; 0, 0, n — 1), around the origin G R 2 . Clearly, 
the space G(R 2 ; 0, 0, n — 1) is homotopy equivalent to C7(R 2 , n) and thus the coho- 
mology algebra H*(G(Tl 2 ; 0, 0, n — 1); k), as given by Proposition 2.2 of [[/J, has 1- 
dimensional generators s\, . . . , s n , which satisfy the relations s 2 = for i = 1, . . . , n 
and also a relation of degree n — 1, cf. formula (4) in [0. From Remark 9 in || we 
obtain that 

n 
i=l 

Let s G H 1 ^ 1 ; k) denote the generator corresponding to the usual anti-clockwise 
orientation of the circle. Then 

g*(si) = s, i = 1,2,..., 7i. (6.6) 

Indeed, = c?jS, where di is the degree of the following map S 1 — ► 5 1 

and hence it is clear that dj = 1. Here i?^ denotes the plane rotation by angle 0. 
Comparing (|6.5|) and (|6.6|) we obtain 



i=l 



where we have used the assumption that n is odd. (Note that for n even the above 
arguments give f*{<J\) = 0.) 

Let us examine the Serre spectral sequence of fibration ( |6.4|) . It has two rows 
and may have one nontrivial differential. Since we know that the fundamental 
class of the fiber s G i? 1 (S' 1 ;k) survives, i.e. application of the differential to it 
gives zero, it follows that all the differentials in the Serre spectral sequence vanish. 
We conclude that the cohomology algebra of the base H*(G(S 2 , n); k) is the factor 
of H*(SO(3) x G(S 2 ; A,A,n— 1); k) with respect to the ideal generated by class 
01. Since o"2i+i = <Ji<J2i, we obtain that H*(G(S 2 , n); k) has generators w with 
degu> = 3 and with deg<T2i = 2i, where % = 0, 1, . . . , (n — 3)/2, which satisfy 
relations fl6.2|). 

□ 
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7. Proof of Theorem 2 

Let T C R m+1 be a compact strictly convex domain with smooth boundary 
X = dT. Consider the smooth function 

i=n 

L x : G(X, n) -> R, L x {x u . . . ,x n ) = \x t - x i+1 \ 

i=i 

(the negative total length), where we understand the indices cyclically modulo n, 
i.e. x n+ i = X\. The critical points of Lx are in 1-1 correspondence with n-periodic 
billiard trajectories in X. 
Fix e > and consider 

n 

G e c G(X, n), G e = {( Xl , ...,x n )Gl xn :[J \x, - x i+1 \ > e}. 

i=i 

According to Proposition 4.1 from [jTJ, if e > is small enough then: 

(a) G € is a compact manifold with boundary; 

(b) the inclusion G t C G(X, n) is a -D n -equivariant homotopy equivalence; 

(c) all critical points of Lx are contained in G e ; 

(d) at every point of dG t the gradient of L x has the outward direction. 

Now we apply Proposition [T7| with M = G t , f = L x , and G = D n . We conclude 
that the number of _D n -orbits of n-periodic billiard trajectories in X is at least 

cat(G £ /D n ) = cat{G(X,n)/D n ). 

Since we assume that n is a odd prime, the action of D n on G(X, n) is free, and 
we may use inequality ( |5.2| ) which gives 



cat(G(X,n)/D n ) > cat(G(X,n)) > cl(G(S m ,n)) + 1. 

Theorems |1| and [19| allow to estimate cat(G(S m ,n). Assume first that m is 
odd, m > 1. Then (according to Theorem ^) we have a nonzero cup-product 



which shows that the cup-length of G(S m , n) for odd m > 1 is at least n — 1. This 
gives lower bound n on the number of D n -orbits of n-periodic billiard trajectories 
in X for m odd. 

If m is even, then (by Theorem [19]) we have a nonzero cup-product 

n — 3 

a^w E H*(G(S m ,n);k), 

where k is a field of characteristic ^ 2. This shows that for even m the cup-length 
of G(S m , n) for even m is at least in — l)/2. This gives lower bound (n + l)/2 on 
the number of _D n -orbits of n-periodic billiard trajectories in X for m even. 

□ 
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